1 Overview

The focus of this document/website is to provide guidance on conducting initial data analysis in a reproducible manner in the context of intended regression analyses.

2 IDA Framework

The IDA framework consists of six steps [Huebner et al 2018, Figure 1], here we assume that metadata (step I) exist in sufficient detail, and that data cleaning (step II) was already performed. Metadata summarize background information about the data to properly conduct IDA steps, and a data cleaning process identifies and corrects technical errors. The data screening (step III) examines data properties to inform decisions about the intended analysis. Initial data reporting (step IV) document insight of the previous steps and can be referred to when interpreting results from the regression modeling. Consequences of these analyses can be that the analysis plan needs to be refined or updated (step V). Finally, reporting of IDA results in research papers (step VI) are necessary to ensure transparency regarding key findings that influence the analysis or interpretation of results. Further details about the elements of IDA are discussed in [TG3 papers].

IDA framework

IDA framework

References

Huebner M, le Cessie S, Schmidt CO, Vach W . A contemporary conceptual framework for initial data analysis. Observational Studies 2018; 4: 171-192. Link

Huebner M, Vach W, le Cessie S, Schmidt C, Lusa L. Hidden Analyses: a review of reporting practice and recommendations for more transparent reporting of initial data analyses. BMC Med Res Meth 2020; 20:61. Link

3 Scope of the regression analyses for the examples

Regression models can be used for a wide range of purposes, for the purpose of these examples the assumptions on the regression analysis set-up in this paper are listed in Table 1. Thus, IDA tasks will be explained in a well-defined, practically relevant setting. Since a key principle is that IDA does not touch the research question no associations between dependent (outcome) and independent (non-outcome) variables are considered.

Table 1: The scope of the regression analyses considered for IDA tasks

Aspects of the research plan Assumptions in this paper Reason for the assumption
Dependent (outcome) variable One dependent variable that can be continuous or binary; exclude time-to-event or longitudinal outcomes Explain IDA tasks in a well-defined, practically relevant setting
Regression models Models with linear predictors Explain IDA tasks in a well-defined, practically relevant setting
Purpose of regression model Adjust effect of one variable of interest for confounders; quantify the effects of explanatory variables on the outcome Explain IDA tasks in a well-defined, practically relevant setting
Independent variables “explanatory” or “confounder” depending on purpose of model; small to moderate number of mixed types; Not high dimensional; no repeated measurements To demonstrate IDA approaches for a mix of variables likely to be encountered in practice
Statistical analysis plan Exists, defines the outcome variable, the type of regression model to be used, and a set of independent variables IDA does not touch the research question, but may lead to an update or refinement of the analysis plan

References:

Vach W. Regression Models as a Tool in Medical Research. Chapman/Hall CRC 2012

Harrell FE. Regression Modeling Strategies. Springer (2nd ed) 2015

Royston P and Sauerbrei W. Multivariable Model Building. Wiley (2008)

[…]

4 Data screening and possible actions

4.1 Univariate distributions

What to look at Possible actions: Interpretation Possible actions: SAP Possible actions: Presentation
Continuous variables General skewness Help in interpreting results Update SAP Update intended presentation of results
Continuous variables General skewness Wide CI for coefficients Use variable as log-transformed Update intended presentation of results
Continuous variables Outliers Disproportional impact on results Winsorize or transform Model involves winsorization
Continuous variables Spike at 0 Narrow CI at 0 Use appropriate representation of variable in model Use 2 (or more) coefficients to distinguish 0 from non-0 continuous part
Categorical variables Frequencies Comparisons to default reference probably irrelevant Change reference category Contrasts compare to (new) reference category
Categorical variables Rare categories Wide CI for coefficients Collapse/exclude Fewer categories to present
Categorical variables One very frequent category Comparisons irrelevant? Exclude variable Variable omitted

4.2 Bivariate distributions

What to look at Possible actions: Interpretation Possible actions: SAP Possible actions: Presentation
Continuous by continuous Outliers (from the cloud) Disproportional impact on results Winsorize or transform Model involves winsorization
Continuous by continuous Correlations Wide CI for coefficients Winsorize or transform Model involves winsorization
Continuous by categorical Outliers (only visible in bivariate plot) Wide CI for coefficients
Categorical by categorical Frequent/rare combinations Comparison to default reference irrelevant Change reference category Contrasts compare to (new) reference category
Categorical by categorical Frequent/rare combinations interactions relevant? Remove interaction from model Fewer interactions to present

4.3 Missing values

What to look at Possible actions: Interpretation Possible actions: SAP Possible actions: Presentation
Per variable Number and proportion Wide CI for coefficients Remove variable if many missing values
Pattern Variables missing independently or together Omit variables together Changes model
Pattern Variables missing dependent on levels of other variables Systematic missingness? Model still based on representative? IPW needed? Weighted analysis
Complete cases Number and proportion Few cases left for main CCO analysis Multiple imputation (or other way of dealing with missing values)? Result from MI analysis? Or applicability restricted to a subpopulation?

References

Huebner M, le Cessie S, Schmidt CO, Vach W . A contemporary conceptual framework for initial data analysis. Observational Studies 2018; 4: 171-192. Link

Harrell FE. Regression Modeling Strategies. Springer (2nd ed) 2015

[…]

CRASH-2

5 Introduction to CRASH-2

Since a key principle of IDA is not to touch the research questions, before IDA commences the research aim and statistical analysis plan need to be in place. IDA may lead to an update or refinement of the analysis plan. To demonstrate the workflow and content of IDA, we created a hypothetical research aim and corresponding statistical analysis plan, which is described in more detail in the section Crash2_SAP.Rmd.

Hypothetical research aim for IDA is to develop a multivariable model for early death (death within 28 days from injury) using nine independent variables of mixed type (continuous, categorical, semicontinuous) with the primary aim of prediction and a secondary aim of describing the association of each variable with the outcome.

A prediction model was developed and validated based on this data set in “Predicting early death in patients with traumatic bleeding” Perel et al, BMJ 2012, [supplement available at]. The assumed research aim is in line with the prediction model

5.1 CRASH-2 Description

Clinical Randomisation of an Antifibrinolyticin Significant Haemorrhage(CRASH-2) was a large randomised placebo controlled trial among trauma patients with, or at risk of, significant haemorrhage, of the effects of antifibrinolytic treatment on death and transfusion requirement. The study is described at the original trial website. A public version of the data set is found at a repository of public data sets hosted by the Vanderbilt University’s Department of Biostatistics (Prof. Frank Harrell Jr.).

The data set includes 20,207 patients and 44 variables.

Note: In contrast to the analysis described in Perel et al, variables describing the economic region and the treatment allocation are missing in the public version of the data set, and while the data set contains 20,207 patients, the research paper mentions 20,127 patients having been included in the study.

5.2 Crash2 dataset contents

5.2.1 Source dataset

We refer to the source data set as the dataset available online here

Display the source dataset contents. This dataset is in the data-raw folder of the project directory.


Data frame:crash2

20207 observations and 44 variables, maximum # NAs:17121  
NameLabelsUnitsLevelsClassStorageNAs
entryidUnique Numbers for Entry Formsintegerinteger 0
sourceMethod of Transmission of Entry Form to CC 5integer 0
trandomisedDate of RandomizationDatedouble 0
outcomeidUnique Number From Outcome Databaseintegerinteger 80
sex 2integer 1
ageinteger 4
injurytimeHours Since Injurynumericdouble 11
injurytype 3integer 0
sbpSystolic Blood PressuremmHgintegerinteger 320
rrRespiratory Rate/minintegerinteger 191
ccCentral Capillary Refille Timesintegerinteger 611
hrHeart Rate/minintegerinteger 137
gcseyeGlasgow Coma Score Eye Openingintegerinteger 732
gcsmotorGlasgow Coma Score Motor Responseintegerinteger 732
gcsverbalGlasgow Coma Score Verbal Responseintegerinteger 735
gcsGlasgow Coma Score Totalintegerinteger 23
ddeathDate of DeathDatedouble17121
causeMain Cause of Death 7integer17118
scauseotherDescription of Other Cause of Death227integer 0
statusStatus of Patient at Outcome if Alive 3integer 3169
ddischargeDate of discharge, transfer to other hospital or day 28 from randomizationDatedouble 3185
conditionCondition of Patient at Outcome if Alive 5integer 3251
ndaysicuNumber of Days Spent in ICUnumericdouble 182
bheadinjSignificant Head Injuryintegerinteger 80
bneuroNeurosurgery Doneintegerinteger 80
bchestChest Surgery Doneintegerinteger 80
babdomenAbdominal Surgery Doneintegerinteger 80
bpelvisPelvis Surgery Doneintegerinteger 80
bpePulmonary Embolismintegerinteger 80
bdvtDeep Vein Thrombosisintegerinteger 80
bstrokeStrokeintegerinteger 80
bbleedSurgery for Bleedingintegerinteger 80
bmiMyocardial Infarctionintegerinteger 80
bgiGastrointestinal Bleedingintegerinteger 80
bloadingComplete Loading Dose of Trial Drug Givenintegerinteger 80
bmaintComplete Maintenance Dose of Trial Drug Givenintegerinteger 80
btransfBlood Products Transfusionintegerinteger 80
ncellNumber of Units of Red Call Products Transfusednumericdouble 9963
nplasmaNumber of Units of Fresh Frozen Plasma Transfusedintegerinteger 9964
nplateletsNumber of Units of Platelets Transfusedintegerinteger 9964
ncryoNumber of Units of Cryoprecipitate Transfusedintegerinteger 9964
bviiRecombinant Factor VIIa Givenintegerinteger 374
boxidTreatment Box Numberintegerinteger 0
packnumTreatment Pack Numberintegerinteger 0

VariableLevels
sourcetelephone
telephone entered manually
electronic CRF by email
paper CRF enteredd in electronic CRF
electronic CRF
sexmale
female
injurytypeblunt
penetrating
blunt and penetrating
causebleeding
head injury
myocardial infarction
stroke
pulmonary embolism
multi organ failure
other
scauseother
Acute Hypoxia
ACUTE LUNG INJURY
Acute Pulmonary Oedema
Acute Renal Failure
ACUTE RESPIRATORY DISTRESS SYNDROME (ARDS)
acute respiratory failure
acute respiratory failure+sepsis
air amboli (embolism)
Air embolism caused by penetrating lung trauma
...
statusdischarged
still in hospital
transferred to other hospital
conditionno symptoms
minor symptoms
some restriction in lifestyle but independent
dependent, but not requiring constant attention
fully dependent, requiring attention day and night

5.2.2 Updated analysis dataset

Additional meta-data is added to the original source data set. We write this new modified data set back to the data folder after adding additional meta-data for the following variables:

  • age - add label “Age” and unit “years”.
  • injury time - add unit “hours”.
  • total Glasgow coma score - add unit “points”.

At the stage we select the variables of interest to take in to the IDA phase by dropping variables we do not check in IDA.

As a cross check we display the contents again to ensure the additional data is added, and then write back the changes to the data folder in the file “data/a_crash2.rds”.

Input object size: 1221480 bytes; 12 variables 20207 observations New object size: 1223272 bytes; 12 variables 20207 observations Input object size: 1546808 bytes; 14 variables 20207 observations New object size: 1385720 bytes; 14 variables 20207 observations


Data frame:a_crash2

20207 observations and 14 variables, maximum # NAs:17121  
NameLabelsUnitsLevelsClassStorageNAs
entryidUnique Numbers for Entry Formsintegerinteger 0
trandomisedDate of RandomizationDatedouble 0
ddeathDate of DeathDatedouble17121
ageAgeyearsintegerinteger 4
sexSex2integer 1
sbpSystolic Blood PressuremmHgintegerinteger 320
hrHeart Rate/minintegerinteger 137
rrRespiratory Rate/minintegerinteger 191
gcsGlasgow Coma Score Totalpointsintegerinteger 23
ccCentral Capillary Refille Timesintegerinteger 611
injurytimeHours Since Injuryhoursnumericdouble 11
injurytypeInjury type3integer 0
time2deathinteger17121
earlydeathDeath within 28 days from injuryintegerinteger 0

VariableLevels
sexmale
female
injurytypeblunt
penetrating
blunt and penetrating

5.3 Section session info

## R version 3.6.1 (2019-07-05)
## Platform: x86_64-w64-mingw32/x64 (64-bit)
## Running under: Windows 10 x64 (build 17763)
## 
## Matrix products: default
## 
## locale:
## [1] LC_COLLATE=English_United States.1252 
## [2] LC_CTYPE=English_United States.1252   
## [3] LC_MONETARY=English_United States.1252
## [4] LC_NUMERIC=C                          
## [5] LC_TIME=English_United States.1252    
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] Hmisc_4.4-0     Formula_1.2-3   survival_3.2-3  lattice_0.20-40
##  [5] forcats_0.5.0   stringr_1.4.0   dplyr_0.8.5     purrr_0.3.4    
##  [9] readr_1.3.1     tidyr_1.0.2     tibble_3.0.1    ggplot2_3.3.0  
## [13] tidyverse_1.3.0 here_0.1       
## 
## loaded via a namespace (and not attached):
##  [1] Rcpp_1.0.4.6        lubridate_1.7.4     png_0.1-7          
##  [4] assertthat_0.2.1    rprojroot_1.3-2     digest_0.6.25      
##  [7] R6_2.4.1            cellranger_1.1.0    backports_1.1.7    
## [10] acepack_1.4.1       reprex_0.3.0        evaluate_0.14      
## [13] httr_1.4.1          pillar_1.4.4        rlang_0.4.6        
## [16] readxl_1.3.1        data.table_1.12.8   rstudioapi_0.11    
## [19] rpart_4.1-15        Matrix_1.2-18       checkmate_2.0.0    
## [22] rmarkdown_2.1       splines_3.6.1       foreign_0.8-76     
## [25] htmlwidgets_1.5.1   munsell_0.5.0       broom_0.5.5        
## [28] compiler_3.6.1      modelr_0.1.6        xfun_0.12          
## [31] pkgconfig_2.0.3     base64enc_0.1-3     htmltools_0.4.0    
## [34] nnet_7.3-13         tidyselect_1.1.0    htmlTable_1.13.3   
## [37] gridExtra_2.3       bookdown_0.18       fansi_0.4.1        
## [40] crayon_1.3.4        dbplyr_1.4.2        withr_2.2.0        
## [43] grid_3.6.1          nlme_3.1-145        jsonlite_1.6.1     
## [46] gtable_0.3.0        lifecycle_0.2.0     DBI_1.1.0          
## [49] magrittr_1.5        scales_1.1.1        rmdformats_0.3.7   
## [52] cli_2.0.2           stringi_1.4.6       fs_1.3.2           
## [55] latticeExtra_0.6-29 xml2_1.2.5          ellipsis_0.3.0     
## [58] generics_0.0.2      vctrs_0.3.0         RColorBrewer_1.1-2 
## [61] tools_3.6.1         glue_1.4.1          hms_0.5.3          
## [64] jpeg_0.1-8.1        yaml_2.2.1          colorspace_1.4-1   
## [67] cluster_2.1.0       rvest_0.3.5         knitr_1.28         
## [70] haven_2.2.0

6 Statistical analysis plan

Since a key principle of IDA is not to touch the research questions, before IDA commences the research aim and statistical analysis plan needs to be in place. IDA may lead to an update or refinement of the analysis plan. To demonstrate the workflow and content of IDA, we created a hypothetical research aim and corresponding statistical analysis plan.

Hypothetical research aim for IDA: Develop a multivariable model for early death (death within 28 days from injury) using nine independent variables of mixed type (continuous, categorical, semicontinuous) with the primary aim of prediction and a secondary aim of describing the association of each variable with the outcome.

The assumed analysis aim is in line with the prediction model presented by Perel et al, BMJ 2012, supplement available at.

6.1 Outcome variable

Early death, i.e. in-hospital death within 28 days from injury (binary variable)

6.2 Statistical methods

Logistic regression will be used to model early death by the following independent variables (measured at randomisation) deemed important to predict early death.

Demographic measurements:

  • Age (age, years)
  • Sex (sex, male or female)

Physiological measurements:

  • Systolic blood pressure (sbp, mmHg)
  • Heart rate (hr, 1/min)
  • Respiratory rate (rr, 1/min)
  • Glasgow coma score (gcs, points)
  • Central capillary refill time (cc, seconds)

Characteristics of injury measurements:

  • Time since injury (injurytime, hours)
  • Type of injury (injurytype, ‘blunt’, ‘penetrating’ or ‘blunt and penetrating’)

Restricted cubic splines with 3 degrees of freedom with knots set to default values will be used for continuous variables. As the final prediction model should be parsimonious enough to simplify its application, a backward elimination algorithm with a significance level set at \(\alpha=0.05\) will be applied to remove statistically insignificant effects. Finally, nonlinear representation of each continuous variable will be tested against linear representation at \(\alpha=0.05\). In case of lacking added value of a nonlinear effect, the model will be refitted with a linear effect for that variable.

6.3 Remarks

  • Regarding type of injury, the original paper describes its treatment in the model as follows: ‘Type of injury had three categories—-penetrating, blunt, or blunt and penetrating—but we analysed it as ’penetrating’ or ‘blunt and penetrating.’ ’ It is not clear from that description what happened to the ‘blunt’ group. (I assume they were collapsed with ‘blunt and penetrating’.) ** we are going to consider the three categories, and then check aout recommendations for the final analysis-MH**

  • The original paper describes the modeling approach as follows: ‘We used a backward step-wise approach. Firstly, we included all potential prognostic factors and interaction terms that users considered plausible. These interactions included all potential predictors with type of injury, time since injury, and age. We then removed, one at a time, terms for which we found no strong evidence of an association, judged according to the P values (<0.05) from the Wald test.’ This would mean they tested at least 24 interaction terms, each possibly using several degrees of freedom! In the final model, only an interaction of Glasgow coma score and type of injury was included.

6.4 Preparations

The outcome variable, early death (i.e., death within 28 days from injury) must be computed from the time span between date of death and date of randomization using the following logic:

  • transform ddeath and trandomisation into an interpretable date format and then compute the difference
  • interpret missing (i.e. NAs) as ‘not died within study period, at least not within 28 days’
  • if patients died after 28 days, treat as alive

This can be derived using the following code logic:

We also display the marginal distribution of the derived outcome variable.

Characteristic N = 202071
Death within 28 days from injury 3076 (15%)

1 Statistics presented: n (%)

The number of deaths computed in the data set coincides with the number reported in Perel et al, BMJ 2012.

6.5 Sources

Data obtained from http://biostat.mc.vanderbilt.edu/wiki/Main/DataSets

To download the data set, click the link to data set

6.5.1 Data dictionary

The data dictionary can be found LINK

6.6 References

CRASH-2 Collaborators. Effects of tranexamic acid on death, vascular occlusive events, and blood transfusion in trauma patients with significant haemorrhage (CRASH-2): a randomised, placebo-controlled trial. Lancet 2010;376:23-32

Perel P, Prieto-Merino D, Shakur H, Clayton T, Lecky F, Bouamra O, Russell R, Faulkner M, Steyerberg EW, Roberts I. Predicting early death in patients with traumatic bleeding: development and validation of prognostic model. BMJ 2012; 345(aug15 1): e5166.

7 Univariate distribution checks

This section reports a series of univariate summary checks of the CRASH-2 dataset.

7.1 Data set overview

Using the Hmisc describe function, we provide an overview of the data set. The descriptive report also provides histograms of continuous variables. For ease of scanning the information, we group the report by measurement type.

7.1.1 Demographic variables

Demographic variables

2 Variables   20207 Observations

age: Age years
image
nmissingdistinctInfoMeanGmd.05.10.25.50.75.90.95
202034840.99934.5615.5518192430435564
lowest : 1 14 15 16 17 , highest: 92 94 95 96 99
sex: Sex
nmissingdistinct
2020612
 Value        male female
 Frequency   16935   3271
 Proportion  0.838  0.162
 

7.1.2 Physiological measurements

Physiological measurements

5 Variables   20207 Observations

sbp: Systolic Blood Pressure mmHg
image
nmissingdistinctInfoMeanGmd.05.10.25.50.75.90.95
198873201730.98998.4527.86 60 70 80 95110130143
lowest : 4 10 12 20 25 , highest: 225 230 234 240 250
hr: Heart Rate /min
image
nmissingdistinctInfoMeanGmd.05.10.25.50.75.90.95
200701371730.996104.523.38 70 80 90105120130140
lowest : 3 4 5 6 10 , highest: 190 192 198 200 220
rr: Respiratory Rate /min
image
nmissingdistinctInfoMeanGmd.05.10.25.50.75.90.95
20016191680.9923.067.05214162022263035
lowest : 1 2 3 4 5 , highest: 90 91 94 95 96
gcs: Glasgow Coma Score Total points
image
nmissingdistinctInfoMeanGmd.05.10.25.50.75.90.95
2018423130.86312.473.594 4 61115151515
lowest : 3 4 5 6 7 , highest: 11 12 13 14 15
 Value          3     4     5     6     7     8     9    10    11    12    13    14
 Frequency    784   520   441   584   733   576   504   663   586   951  1356  2140
 Proportion 0.039 0.026 0.022 0.029 0.036 0.029 0.025 0.033 0.029 0.047 0.067 0.106
                 
 Value         15
 Frequency  10346
 Proportion 0.513
 

cc: Central Capillary Refille Time s
image
nmissingdistinctInfoMeanGmd.05.10.25.50.75.90.95
19596611200.9453.2671.671223456
lowest : 1 2 3 4 5 , highest: 17 18 20 30 60
 Value          1     2     3     4     5     6     7     8     9    10    11    12
 Frequency   1510  5328  6020  3367  1805   802   268   271    45   139     3     7
 Proportion 0.077 0.272 0.307 0.172 0.092 0.041 0.014 0.014 0.002 0.007 0.000 0.000
                                                           
 Value         13    15    16    17    18    20    30    60
 Frequency      3    19     3     1     1     2     1     1
 Proportion 0.000 0.001 0.000 0.000 0.000 0.000 0.000 0.000
 

7.1.3 Characteristics of injury

Characteristics of injury

2 Variables   20207 Observations

injurytime: Hours Since Injury hours
image
nmissingdistinctInfoMeanGmd.05.10.25.50.75.90.95
2019611930.9722.8442.350.51.01.02.04.06.07.0
lowest : 0.10 0.15 0.20 0.25 0.30 , highest: 22.00 45.00 48.00 72.00 96.00
injurytype: Injury type
image
nmissingdistinct
2020703
 Value                      blunt           penetrating blunt and penetrating
 Frequency                  11189                  6552                  2466
 Proportion                 0.554                 0.324                 0.122
 

7.2 Categorical variables

We now provide a closer visual examination of the categorical predictors.

7.2.1 Categorical ordinal plots

The Glasgow coma score, an ordinal categorical variable, is also displayed separately.

7.3 Continuous variables

A closer visual examination of continuous predictors.

There is evidence of digit preference. Explore further with targeted summaries. A more detailed univariate summaries for the variables of interest are also provided below.

7.3.1 Age

Distribution of subject age [years]

Distribution of subject age [years]

Five patients under the age of 17, the inclusion criteria for the study, with one patient aged 1.

7.3.2 Blood pressure

Distribution of SBP

Distribution of SBP

7.3.3 Respiratory rate

Distribution of respiratory rate

Distribution of respiratory rate

7.3.4 Heart rate

Distribution of heart rate

Distribution of heart rate

7.3.5 Central capillary refill time

Distribution of Central capillary refill time

Distribution of Central capillary refill time

7.3.6 Hours since injury

Distribution of hours since injury

Distribution of hours since injury

7.4 Section session info

## R version 3.6.1 (2019-07-05)
## Platform: x86_64-w64-mingw32/x64 (64-bit)
## Running under: Windows 10 x64 (build 17763)
## 
## Matrix products: default
## 
## locale:
## [1] LC_COLLATE=English_United States.1252 
## [2] LC_CTYPE=English_United States.1252   
## [3] LC_MONETARY=English_United States.1252
## [4] LC_NUMERIC=C                          
## [5] LC_TIME=English_United States.1252    
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] Hmisc_4.4-0     Formula_1.2-3   survival_3.2-3  lattice_0.20-40
##  [5] forcats_0.5.0   stringr_1.4.0   dplyr_0.8.5     purrr_0.3.4    
##  [9] readr_1.3.1     tidyr_1.0.2     tibble_3.0.1    ggplot2_3.3.0  
## [13] tidyverse_1.3.0 here_0.1       
## 
## loaded via a namespace (and not attached):
##  [1] httr_1.4.1          jsonlite_1.6.1      splines_3.6.1      
##  [4] modelr_0.1.6        assertthat_0.2.1    highr_0.8          
##  [7] latticeExtra_0.6-29 cellranger_1.1.0    yaml_2.2.1         
## [10] pillar_1.4.4        backports_1.1.7     glue_1.4.1         
## [13] digest_0.6.25       RColorBrewer_1.1-2  checkmate_2.0.0    
## [16] rvest_0.3.5         colorspace_1.4-1    htmltools_0.4.0    
## [19] Matrix_1.2-18       pkgconfig_2.0.3     broom_0.5.5        
## [22] haven_2.2.0         bookdown_0.18       patchwork_1.0.0    
## [25] scales_1.1.1        jpeg_0.1-8.1        htmlTable_1.13.3   
## [28] farver_2.0.3        generics_0.0.2      ellipsis_0.3.0     
## [31] withr_2.2.0         nnet_7.3-13         cli_2.0.2          
## [34] magrittr_1.5        crayon_1.3.4        readxl_1.3.1       
## [37] evaluate_0.14       fs_1.3.2            fansi_0.4.1        
## [40] nlme_3.1-145        xml2_1.2.5          foreign_0.8-76     
## [43] tools_3.6.1         data.table_1.12.8   hms_0.5.3          
## [46] lifecycle_0.2.0     munsell_0.5.0       reprex_0.3.0       
## [49] cluster_2.1.0       compiler_3.6.1      rlang_0.4.6        
## [52] grid_3.6.1          rstudioapi_0.11     htmlwidgets_1.5.1  
## [55] base64enc_0.1-3     labeling_0.3        rmarkdown_2.1      
## [58] gtable_0.3.0        DBI_1.1.0           R6_2.4.1           
## [61] gridExtra_2.3       lubridate_1.7.4     knitr_1.28         
## [64] rprojroot_1.3-2     stringi_1.4.6       rmdformats_0.3.7   
## [67] Rcpp_1.0.4.6        vctrs_0.3.0         rpart_4.1-15       
## [70] acepack_1.4.1       png_0.1-7           dbplyr_1.4.2       
## [73] tidyselect_1.1.0    xfun_0.12

8 Multivariate distributions

8.1 Overview

8.1.2 Variable clustering

Variable clustering is used for assessing collinearity, redundancy, and for separating variables into clusters that can be scored as a single variable, thus resulting in data reduction.

## Hmisc::varclus(x = ~age + sbp + hr + rr + cc + gcs + injurytime + 
##     injurytype + sex, data = a_crash2)
## 
## 
## Similarity matrix (Spearman rho^2)
## 
##                                  age  sbp   hr   rr   cc  gcs injurytime
## age                             1.00 0.00 0.00 0.00 0.00 0.00       0.01
## sbp                             0.00 1.00 0.11 0.03 0.07 0.01       0.01
## hr                              0.00 0.11 1.00 0.05 0.02 0.02       0.00
## rr                              0.00 0.03 0.05 1.00 0.02 0.00       0.00
## cc                              0.00 0.07 0.02 0.02 1.00 0.02       0.00
## gcs                             0.00 0.01 0.02 0.00 0.02 1.00       0.01
## injurytime                      0.01 0.01 0.00 0.00 0.00 0.01       1.00
## injurytypepenetrating           0.02 0.00 0.01 0.00 0.00 0.06       0.05
## injurytypeblunt and penetrating 0.00 0.01 0.01 0.00 0.00 0.01       0.00
## sexfemale                       0.01 0.00 0.00 0.00 0.00 0.00       0.00
##                                 injurytypepenetrating
## age                                              0.02
## sbp                                              0.00
## hr                                               0.01
## rr                                               0.00
## cc                                               0.00
## gcs                                              0.06
## injurytime                                       0.05
## injurytypepenetrating                            1.00
## injurytypeblunt and penetrating                  0.07
## sexfemale                                        0.02
##                                 injurytypeblunt and penetrating sexfemale
## age                                                        0.00      0.01
## sbp                                                        0.01      0.00
## hr                                                         0.01      0.00
## rr                                                         0.00      0.00
## cc                                                         0.00      0.00
## gcs                                                        0.01      0.00
## injurytime                                                 0.00      0.00
## injurytypepenetrating                                      0.07      0.02
## injurytypeblunt and penetrating                            1.00      0.00
## sexfemale                                                  0.00      1.00
## 
## No. of observations used for each pair:
## 
##                                   age   sbp    hr    rr    cc   gcs injurytime
## age                             20203 19884 20066 20012 19593 20180      20193
## sbp                             19884 19887 19795 19750 19316 19883      19877
## hr                              20066 19795 20070 19943 19482 20066      20059
## rr                              20012 19750 19943 20016 19454 20014      20008
## cc                              19593 19316 19482 19454 19596 19595      19588
## gcs                             20180 19883 20066 20014 19595 20184      20173
## injurytime                      20193 19877 20059 20008 19588 20173      20196
## injurytypepenetrating           20203 19887 20070 20016 19596 20184      20196
## injurytypeblunt and penetrating 20203 19887 20070 20016 19596 20184      20196
## sexfemale                       20202 19886 20069 20015 19595 20183      20195
##                                 injurytypepenetrating
## age                                             20203
## sbp                                             19887
## hr                                              20070
## rr                                              20016
## cc                                              19596
## gcs                                             20184
## injurytime                                      20196
## injurytypepenetrating                           20207
## injurytypeblunt and penetrating                 20207
## sexfemale                                       20206
##                                 injurytypeblunt and penetrating sexfemale
## age                                                       20203     20202
## sbp                                                       19887     19886
## hr                                                        20070     20069
## rr                                                        20016     20015
## cc                                                        19596     19595
## gcs                                                       20184     20183
## injurytime                                                20196     20195
## injurytypepenetrating                                     20207     20206
## injurytypeblunt and penetrating                           20207     20206
## sexfemale                                                 20206     20206
## 
## hclust results (method=complete)
## 
## 
## Call:
## hclust(d = as.dist(1 - x), method = method)
## 
## Cluster method   : complete 
## Number of objects: 10

Plot associations.

8.1.3 Variable redundancy

Redundancy analysis of predictor variables.

## 
## Redundancy Analysis
## 
## Hmisc::redun(formula = ~hr + rr + age + sbp + injurytype + sex, 
##     data = a_crash2)
## 
## n: 19689     p: 6    nk: 3 
## 
## Number of NAs:    518 
## Frequencies of Missing Values Due to Each Variable
##         hr         rr        age        sbp injurytype        sex 
##        137        191          4        320          0          1 
## 
## 
## Transformation of target variables forced to be linear
## 
## R-squared cutoff: 0.9    Type: ordinary 
## 
## R^2 with which each variable can be predicted from all other variables:
## 
##         hr         rr        age        sbp injurytype        sex 
##      0.116      0.044      0.052      0.099      0.061      0.035 
## 
## No redundant variables

8.2 Summary reports by sex

8.2.1 Overall

Baseline characteristics by sex.
N
male
N=16935
female
N=3271
Age
years
20203 23.0 30.0 41.0
33.7 ± 13.6
25.0 35.0 50.0
38.8 ± 16.8
Systolic Blood Pressure
mmHg
19887 80.0 95.0 110.0
98.8 ±  25.5
80.0 90.0 110.0
96.7 ±  25.7
Heart Rate
/min
20070 90.0 105.0 120.0
104.3 ±  21.2
92.0 106.0 120.0
105.2 ±  21.0
Respiratory Rate
/min
20016 20.00 22.00 26.00
23.07 ±  6.77
20.00 22.00 26.00
23.03 ±  6.58
Central Capillary Refille Time
s
19596 2.00 3.00 4.00
3.27 ± 1.72
2.00 3.00 4.00
3.23 ± 1.59
Glasgow Coma Score Total
points
20184 11.00 15.00 15.00
12.44 ±  3.72
12.00 14.00 15.00
12.62 ±  3.46
Hours Since Injury
hours
20196 1.00 2.00 4.00
2.85 ± 2.39
1.00 2.00 4.00
2.84 ± 2.67
Injury type : blunt 20207 0.53 8962/16935 0.68 2227/ 3271
  penetrating 0.35 5930/16935 0.19 621/ 3271
  blunt and penetrating 0.12 2043/16935 0.13 423/ 3271
a b c represent the lower quartile a, the median b, and the upper quartile c for continuous variables. x ± s represents X ± 1 SD.   N is the number of non-missing values.

8.2.2 Distribution of age by sex

Distribution of age by sex

8.2.3 Distribution of systolic blood pressure by sex

Distribution of systolic blood pressure by sex

8.2.4 Distribution of heart rate by sex

Distribution of heart rate by sex

8.2.5 Distribution of respiratory rate by sex

Distribution of respiratory rate by sex

8.2.6 Distribution of central capillary refille time by sex

Distribution of centrail capillary refille time by sex

8.2.7 Distribution of hours since injury by sex

Distribution of hours since injury by sex

8.2.8 Distribution of Glasgow comma score by sex

Distribution of Glasgow comma score (point scale) by sex

Distribution of Glasgow comma score (point scale) by sex

8.2.9 Distribution of injury type by sex

Distribution of injury type by sex

Distribution of injury type by sex

8.3 Summary reports by age

Categorize age for the purposes of exploring the relationship between age and other baseline variables. This is purely for exploratory purposes only, and not to influence the analysis strategy to pursue the dichotomization of age.

Characteristic N = 202071
age_C
<30 9070 (45%)
30-44 6477 (32%)
45-59 3204 (16%)
60+ 1452 (7.2%)
NA 4 (<0.1%)

1 Statistics presented: n (%)

Report all variables by age category.

Baseline characteristics by age categories.
N
<30
N=9070
30-44
N=6477
45-59
N=3204
60+
N=1452
Sex : female 20202 0.13 1183/9070 0.15 959/6476 0.21 659/3204 0.32 469/1452
Systolic Blood Pressure
mmHg
19884 80.0 96.0 110.0
98.1 ±  23.8
80.0 90.0 110.0
97.7 ±  25.3
80.0 94.0 112.0
100.1 ±  28.4
80.0 90.0 110.0
100.4 ±  30.2
Heart Rate
/min
20066 91.0 106.0 120.0
105.3 ±  21.3
90.0 106.0 120.0
104.7 ±  20.9
90.0 104.0 120.0
103.3 ±  21.0
88.0 100.0 116.0
101.0 ±  21.8
Respiratory Rate
/min
20012 20.00 22.00 26.00
22.93 ±  6.74
20.00 22.00 26.00
23.24 ±  6.68
20.00 22.00 26.00
23.11 ±  6.80
20.00 22.00 26.00
23.04 ±  6.89
Central Capillary Refille Time
s
19593 2.00 3.00 4.00
3.20 ± 1.77
2.00 3.00 4.00
3.27 ± 1.65
2.00 3.00 4.00
3.34 ± 1.64
2.00 3.00 4.00
3.48 ± 1.56
Glasgow Coma Score Total
points
20180 11.00 15.00 15.00
12.64 ±  3.61
11.00 14.50 15.00
12.39 ±  3.72
11.00 14.00 15.00
12.38 ±  3.70
10.00 14.00 15.00
12.00 ±  3.82
Hours Since Injury
hours
20193 1.00 2.00 4.00
2.71 ± 2.18
1.00 2.00 4.00
2.83 ± 2.28
1.00 2.50 4.50
3.12 ± 3.17
1.00 3.00 4.50
3.12 ± 2.68
Injury type : blunt 20203 0.50 4544/9070 0.53 3462/6477 0.65 2081/3204 0.76 1101/1452
  penetrating 0.38 3448/9070 0.33 2155/6477 0.23 748/3204 0.14 199/1452
  blunt and penetrating 0.12 1078/9070 0.13 860/6477 0.12 375/3204 0.10 152/1452
a b c represent the lower quartile a, the median b, and the upper quartile c for continuous variables. x ± s represents X ± 1 SD.   N is the number of non-missing values.

8.3.1 Distribution of systolic blood pressure by age categories

Distribution of systolic blood pressure by gcs

8.3.2 Distribution of heart rate by age categories

Distribution of heart rate by gcs

8.3.3 Distribution of respiratory rate by age categories

Distribution of respiratory rate by gcs

8.3.4 Distribution of central capillary refille time by age categories

Distribution of centrail capillary refille time by gcs

8.4 Scatter plots with a third or fourth variable

Scatter plot of age and RR by sex and injury type.

Scatter plot of SBP and RR by sex and injury type.

8.5 Baseline characteristics by Glasgow comma score

Baseline characteristics by Glasgow comma score.
N
3
N=784
4
N=520
5
N=441
6
N=584
7
N=733
8
N=576
9
N=504
10
N=663
11
N=586
12
N=951
13
N=1356
14
N=2140
15
N=10346
Age
years
20203 24.0 32.0 44.0
35.5 ± 14.9
25.0 33.0 44.0
35.5 ± 14.1
24.0 32.0 45.0
35.4 ± 14.7
23.0 31.0 45.0
35.4 ± 15.4
23.0 30.0 42.0
33.9 ± 14.0
24.0 32.0 45.0
35.7 ± 15.0
24.0 32.0 44.0
35.5 ± 14.6
24.0 31.0 42.0
34.4 ± 13.8
24.0 33.0 46.0
36.6 ± 15.6
25.0 32.0 45.0
35.9 ± 14.3
25.0 33.0 45.0
36.4 ± 15.0
24.0 31.0 44.0
35.1 ± 14.7
23.0 30.0 41.0
33.7 ± 13.8
Heart Rate
/min
20070 90.0 112.0 128.0
106.9 ±  31.3
95.0 114.0 130.0
110.8 ±  29.2
98.0 110.5 130.0
111.4 ±  25.4
90.0 110.0 123.2
106.2 ±  24.4
95.0 109.0 120.0
107.1 ±  23.0
95.0 110.0 120.0
107.4 ±  24.0
92.0 109.0 120.0
105.5 ±  21.9
96.0 110.0 124.8
108.4 ±  24.0
96.0 110.0 122.0
107.8 ±  20.4
100.0 110.0 122.0
109.3 ±  20.2
96.0 108.0 120.0
106.5 ±  20.1
92.0 105.0 120.0
104.5 ±  19.9
90.0 100.0 115.0
102.0 ±  18.9
Respiratory Rate
/min
20016 12.00 20.00 28.00
20.67 ± 10.74
16.00 22.00 28.00
22.22 ±  9.14
18.00 22.00 28.00
22.89 ±  8.69
18.00 21.00 26.00
22.12 ±  7.56
18.00 20.00 26.00
21.97 ±  7.69
18.00 22.00 28.00
23.11 ±  7.73
20.00 24.00 28.00
23.23 ±  6.99
19.00 22.00 28.00
23.05 ±  6.73
20.00 23.00 28.00
23.45 ±  6.37
20.00 24.00 28.00
24.32 ±  6.41
20.00 22.00 27.00
23.45 ±  6.53
20.00 22.00 26.00
23.41 ±  6.09
20.00 22.00 26.00
23.14 ±  6.07
Systolic Blood Pressure
mmHg
19887 70.0 85.0 103.0
88.7 ±  33.7
78.0 90.0 116.0
96.5 ±  31.2
80.0 90.0 118.0
99.0 ±  30.7
80.0 100.0 127.0
104.3 ±  32.1
80.0 100.0 130.0
105.4 ±  30.6
80.0 90.0 115.0
99.2 ±  29.4
80.0 96.0 120.0
99.6 ±  28.9
80.0 90.0 110.0
92.6 ±  28.0
80.0 90.0 110.0
94.4 ±  26.4
71.0 90.0 100.0
88.4 ±  24.7
80.0 90.0 110.0
95.9 ±  23.5
80.0 90.0 110.0
96.4 ±  22.8
90.0 100.0 110.0
100.5 ±  23.1
Central Capillary Refille Time
s
19596 3.00 4.00 5.00
4.15 ± 2.13
3.00 4.00 5.00
3.84 ± 1.90
2.00 3.00 5.00
3.76 ± 1.91
2.00 3.00 4.00
3.49 ± 1.64
2.00 3.00 4.00
3.28 ± 1.55
2.00 3.00 4.00
3.52 ± 1.69
2.00 3.00 4.00
3.40 ± 3.00
2.00 3.00 4.00
3.37 ± 1.66
2.00 3.00 4.00
3.27 ± 1.51
3.00 3.00 4.00
3.53 ± 1.60
2.00 3.00 4.00
3.40 ± 1.69
2.00 3.00 4.00
3.31 ± 1.73
2.00 3.00 4.00
3.06 ± 1.54
Sex : female 20206 0.14 107/ 784 0.13 68/ 520 0.12 53/ 441 0.16 92/ 584 0.14 100/ 733 0.15 89/ 576 0.15 74/ 504 0.19 124/ 663 0.17 97/ 586 0.21 198/ 951 0.20 270/ 1356 0.18 391/ 2139 0.16 1604/10346
Hours Since Injury
hours
20196 1.00 2.00 4.00
2.54 ± 1.94
1.00 3.00 5.00
3.26 ± 2.20
1.00 3.00 5.00
3.42 ± 2.19
2.00 3.75 6.00
3.75 ± 2.31
2.00 3.00 5.00
3.62 ± 2.20
1.00 3.00 5.00
3.30 ± 2.17
1.00 3.00 5.00
3.12 ± 2.20
1.00 2.00 4.00
3.03 ± 2.19
1.00 2.50 4.00
3.01 ± 2.05
1.00 2.00 4.00
2.75 ± 2.03
1.00 2.00 4.00
2.79 ± 1.97
1.00 2.00 4.00
2.64 ± 1.99
1.00 2.00 4.00
2.71 ± 2.69
Injury type : blunt 20207 0.62 483/ 784 0.71 371/ 520 0.73 324/ 441 0.76 443/ 584 0.76 559/ 733 0.69 399/ 576 0.67 338/ 504 0.61 407/ 663 0.64 377/ 586 0.58 550/ 951 0.60 814/ 1356 0.58 1237/ 2140 0.47 4880/10346
  penetrating 0.22 175/ 784 0.10 53/ 520 0.09 41/ 441 0.10 59/ 584 0.11 77/ 733 0.15 89/ 576 0.17 88/ 504 0.23 151/ 663 0.21 123/ 586 0.29 272/ 951 0.24 326/ 1356 0.29 629/ 2140 0.43 4458/10346
  blunt and penetrating 0.16 126/ 784 0.18 96/ 520 0.17 76/ 441 0.14 82/ 584 0.13 97/ 733 0.15 88/ 576 0.15 78/ 504 0.16 105/ 663 0.15 86/ 586 0.14 129/ 951 0.16 216/ 1356 0.13 274/ 2140 0.10 1008/10346
a b c represent the lower quartile a, the median b, and the upper quartile c for continuous variables. x ± s represents X ± 1 SD.   N is the number of non-missing values.

8.5.1 Distribution of age by Glasgow comma score

Distribution of age by gcs

8.5.2 Distribution of systolic blood pressure by Glasgow comma score

Distribution of systolic blood pressure by gcs

8.5.3 Distribution of heart rate by Glasgow comma score

Distribution of heart rate by gcs

8.5.4 Distribution of respiratory rate by Glasgow comma score

Distribution of respiratory rate by gcs

8.5.5 Distribution of central capillary refille time by Glasgow comma score

Distribution of centrail capillary refille time by gcs

8.6 Section session info

## R version 3.6.1 (2019-07-05)
## Platform: x86_64-w64-mingw32/x64 (64-bit)
## Running under: Windows 10 x64 (build 17763)
## 
## Matrix products: default
## 
## locale:
## [1] LC_COLLATE=English_United States.1252 
## [2] LC_CTYPE=English_United States.1252   
## [3] LC_MONETARY=English_United States.1252
## [4] LC_NUMERIC=C                          
## [5] LC_TIME=English_United States.1252    
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] patchwork_1.0.0 corrplot_0.84   gtsummary_1.2.6 Hmisc_4.4-0    
##  [5] Formula_1.2-3   survival_3.2-3  lattice_0.20-40 plotly_4.9.2.1 
##  [9] forcats_0.5.0   stringr_1.4.0   dplyr_0.8.5     purrr_0.3.4    
## [13] readr_1.3.1     tidyr_1.0.2     tibble_3.0.1    ggplot2_3.3.0  
## [17] tidyverse_1.3.0 here_0.1       
## 
## loaded via a namespace (and not attached):
##  [1] nlme_3.1-145        fs_1.3.2            lubridate_1.7.4    
##  [4] RColorBrewer_1.1-2  httr_1.4.1          rprojroot_1.3-2    
##  [7] tools_3.6.1         backports_1.1.7     R6_2.4.1           
## [10] rpart_4.1-15        DBI_1.1.0           lazyeval_0.2.2     
## [13] colorspace_1.4-1    nnet_7.3-13         withr_2.2.0        
## [16] tidyselect_1.1.0    gridExtra_2.3       compiler_3.6.1     
## [19] cli_2.0.2           rvest_0.3.5         gt_0.2.0.5         
## [22] htmlTable_1.13.3    xml2_1.2.5          sass_0.2.0         
## [25] labeling_0.3        bookdown_0.18       scales_1.1.1       
## [28] checkmate_2.0.0     commonmark_1.7      digest_0.6.25      
## [31] foreign_0.8-76      rmarkdown_2.1       base64enc_0.1-3    
## [34] jpeg_0.1-8.1        pkgconfig_2.0.3     htmltools_0.4.0    
## [37] dbplyr_1.4.2        highr_0.8           htmlwidgets_1.5.1  
## [40] rlang_0.4.6         readxl_1.3.1        rstudioapi_0.11    
## [43] generics_0.0.2      farver_2.0.3        jsonlite_1.6.1     
## [46] crosstalk_1.1.0.1   acepack_1.4.1       magrittr_1.5       
## [49] Matrix_1.2-18       Rcpp_1.0.4.6        munsell_0.5.0      
## [52] fansi_0.4.1         lifecycle_0.2.0     stringi_1.4.6      
## [55] yaml_2.2.1          grid_3.6.1          crayon_1.3.4       
## [58] haven_2.2.0         splines_3.6.1       hms_0.5.3          
## [61] knitr_1.28          pillar_1.4.4        reprex_0.3.0       
## [64] glue_1.4.1          evaluate_0.14       latticeExtra_0.6-29
## [67] data.table_1.12.8   modelr_0.1.6        png_0.1-7          
## [70] vctrs_0.3.0         rmdformats_0.3.7    cellranger_1.1.0   
## [73] gtable_0.3.0        assertthat_0.2.1    xfun_0.12          
## [76] broom_0.5.5         viridisLite_0.3.0   cluster_2.1.0      
## [79] ellipsis_0.3.0

9 Missing data

9.1 Per variable missingness

Number and percentage of missing.

Variable Missing (count) Missing (%)
cc 611 3.02
sbp 320 1.58
rr 191 0.95
hr 137 0.68
gcs 23 0.11
injurytime 11 0.05
age 4 0.02
sex 1 0.00
injurytype 0 0.00

9.2 Missingness patterns over variables

9.3 Complete cases

List out patients with at least one missing value.

Look at the pattern of missing for this set of patients.

9.4 SEction session info

## R version 3.6.1 (2019-07-05)
## Platform: x86_64-w64-mingw32/x64 (64-bit)
## Running under: Windows 10 x64 (build 17763)
## 
## Matrix products: default
## 
## locale:
## [1] LC_COLLATE=English_United States.1252 
## [2] LC_CTYPE=English_United States.1252   
## [3] LC_MONETARY=English_United States.1252
## [4] LC_NUMERIC=C                          
## [5] LC_TIME=English_United States.1252    
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] DT_0.13          kableExtra_1.1.0 gt_0.2.0.5       naniar_0.5.2    
##  [5] Hmisc_4.4-0      Formula_1.2-3    survival_3.2-3   lattice_0.20-40 
##  [9] forcats_0.5.0    stringr_1.4.0    dplyr_0.8.5      purrr_0.3.4     
## [13] readr_1.3.1      tidyr_1.0.2      tibble_3.0.1     ggplot2_3.3.0   
## [17] tidyverse_1.3.0  here_0.1        
## 
## loaded via a namespace (and not attached):
##  [1] nlme_3.1-145        fs_1.3.2            lubridate_1.7.4    
##  [4] webshot_0.5.2       RColorBrewer_1.1-2  httr_1.4.1         
##  [7] UpSetR_1.4.0        rprojroot_1.3-2     tools_3.6.1        
## [10] backports_1.1.7     R6_2.4.1            rpart_4.1-15       
## [13] DBI_1.1.0           colorspace_1.4-1    nnet_7.3-13        
## [16] withr_2.2.0         tidyselect_1.1.0    gridExtra_2.3      
## [19] compiler_3.6.1      cli_2.0.2           rvest_0.3.5        
## [22] htmlTable_1.13.3    xml2_1.2.5          labeling_0.3       
## [25] bookdown_0.18       sass_0.2.0          scales_1.1.1       
## [28] checkmate_2.0.0     commonmark_1.7      digest_0.6.25      
## [31] foreign_0.8-76      rmarkdown_2.1       base64enc_0.1-3    
## [34] jpeg_0.1-8.1        pkgconfig_2.0.3     htmltools_0.4.0    
## [37] dbplyr_1.4.2        htmlwidgets_1.5.1   rlang_0.4.6        
## [40] readxl_1.3.1        rstudioapi_0.11     farver_2.0.3       
## [43] generics_0.0.2      jsonlite_1.6.1      crosstalk_1.1.0.1  
## [46] acepack_1.4.1       magrittr_1.5        Matrix_1.2-18      
## [49] Rcpp_1.0.4.6        munsell_0.5.0       fansi_0.4.1        
## [52] lifecycle_0.2.0     visdat_0.5.3        stringi_1.4.6      
## [55] yaml_2.2.1          plyr_1.8.6          grid_3.6.1         
## [58] crayon_1.3.4        haven_2.2.0         splines_3.6.1      
## [61] hms_0.5.3           knitr_1.28          pillar_1.4.4       
## [64] reprex_0.3.0        glue_1.4.1          evaluate_0.14      
## [67] latticeExtra_0.6-29 data.table_1.12.8   modelr_0.1.6       
## [70] png_0.1-7           vctrs_0.3.0         rmdformats_0.3.7   
## [73] cellranger_1.1.0    gtable_0.3.0        assertthat_0.2.1   
## [76] xfun_0.12           broom_0.5.5         viridisLite_0.3.0  
## [79] cluster_2.1.0       ellipsis_0.3.0